Skip to content

Generalized Modelica URIs#2663

Draft
henrikt-ma wants to merge 32 commits into
masterfrom
MCP/0037
Draft

Generalized Modelica URIs#2663
henrikt-ma wants to merge 32 commits into
masterfrom
MCP/0037

Conversation

@henrikt-ma
Copy link
Copy Markdown
Collaborator

This is MCP-0037 Generalized Modelica URIs. Currently in Draft state to reflect state of being under development.

Comment thread RationaleMCP/0037/ReadMe.md Outdated

The forms containing _relclass_ are referred to as the _lookup-based forms_ (of a Modelica URI). The form with _fullclass_ is referred to as the _fully qualified form_ (of a Modelica URI).

### Lookup-free variation: Staying in the class tree
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced about adding these operations on their own without adding some additional groundwork.
The problem is that when copying classes we either need to make resources absolute (defeating the purpose of this), or copy the resources as well (which leads to duplicated resources). Thus I would propose to not delay this part, and study how it is handled in other systems.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To see the full potential and need for relative class references, one needs to look beyond external resources. A prime example is to refer to one of the figures of the current model from the model's documentation. The figure is stored with the model, and accessing it by a fully qualified path is extremely inconvenient and hard to maintain.

When to copy or reuse external resources is a modeling and tool consideration in my mind. The user needs to put external resources at a suitable level in the package hierarchy with class copying and sharing in mind. A tool should offer good options for what to do with external resources when a class is copied — sometimes the user may prefer cloning the resources, sometimes not, and a helpful tool can fix the broken relative Modelica URIs. A simple tool may not offer anything fancy at all, relying on the modeler to organize external resources and the references to them in such a way that things don't break when a class is copied naively.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To see the full potential and need for relative class references, one needs to look beyond external resources. A prime example is to refer to one of the figures of the current model from the model's documentation. The figure is stored with the model, and accessing it by a fully qualified path is extremely inconvenient and hard to maintain.

But the external resources are generally not stored in that way. In MSL the images are stored in Resources/Images/ and 3d-images in Resources/Data/Shapes.

Having the external data together with the Modelica-file was tried early on, and it failed in several ways in practice.

Obviously we can figure out something better, but my proposal is to not introduce relative paths until we have figured that out.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the external resources are generally not stored in that way. In MSL the images are stored in Resources/Images/ and 3d-images in Resources/Data/Shapes.

The form relative to the encapsulation border is meant to serve this use case:

  • modelica:~?resource=Data/Tables/test_v4.mat (when referenced from within Modelica)

Having the external data together with the Modelica-file was tried early on, and it failed in several ways in practice.

I wasn't there at the time, but I can see that this requires both experienced modelers and intelligent tool support to be a feature that really shines. Of course, allowing non-absolute class references doesn't force anybody to use them for external resources, and they probably won't get much use until there is intelligent support in tools.

Obviously we can figure out something better, but my proposal is to not introduce relative paths until we have figured that out.

But we need them desperately for other things than the external resources…

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we discuss relative path to refer to something other than external resources, e.g., having the documentation of model M referring to a component (a) in its diagram layer then I strongly believe that we shouldn't view it as class-relative - but as a normal reference in some way, so that if we have:

model Z
  M m;
end Z;

and open the documentation for 'm' then the documentation should refer to m.a, and below for z.m it should show z.m.a:

model Z2
  Z z(m(redeclare A2 a));
end Z2;

Similarly if we refer to a parameter a.p from M it should in z.m refer to z.m.a.p.

And some of problems with storing external data with the Modelica-files that were found: it relies on the file-system and is thus not ideal when storing models in a data-base, and if you change how package is stored it looks odd (e.g., when encrypting packages), and it was a bit messy as you cannot easily separate the resources from the rest.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To me it would be ideal if the syntax for this matched the existing lookup in Modelica as much as possible. Having a special "?component=a" seems odd, and it will be difficult to remember what rules apply to class parameters, local lookup etc.

I should have mentioned that we should at least think about how to make best use of the fragment specifier. In the simple case of only referring to a component of the referenced class, it would certainly look nice compared to using the component query:

  • modelica:/Modelica/Constants#pi (compared to: modelica:/Modelica/Constants?component=pi)

However, I'm not sure that this would serve us well in the end, considering the combination of referencing a component, and asking for a resource of that component. Imagine asking for the icon of the component foo.bar of the current class:

  • modelica:?view=icon#foo.bar

Here, one would need to first resolve the fragment specifier, and then apply the view query, even though the query goes before the fragment specifier.

I believe it would be better to always leave the interpretation of the fragment specifier until the query has been resolved, possibly leading to different interpretations depending on context. For example, I think we all expect that the fragment specifier can be used to link to an id or a.name in the documentation:

  • modelica:/Modelica?view=info#ConnectorEquations
  • modelica:?component=foo.bar&view=info#introduction (documentation for the component foo.bar in the current class, at the element introduction)

Right now, I can't think of any other obvious use of the fragment specifier, but it seems like a useful opportunity to be able to give it other meanings in other contexts.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is time to identify some subtopics and start some PRs to add some structure to the discussion…

Copy link
Copy Markdown
Collaborator Author

@henrikt-ma henrikt-ma Oct 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suggested way of using the new URIs in annotations for figures now shows how the fragment specifier could work as a really nice way of referencing a Plot in the context of a given Figure. I think this is just the result of having developed a better understanding of how the fragment specifier is meant to be used: the power of the fragment specifier is really in its application relative to a resource specified by a query. Therefore, when allowing more things with identifier inside a Figure in the future, one should make sure that the identifiers are still unique within the Figure, as that will enable making good use of the fragment specifier.

Applying this reasoning to the references to components seems to indicate that the fragment specifier could work in some situations by simply thinking of the components as being resources identified by their component reference relative to a class. On the other hand, it isn't going to work in all situations as, for example, the fragment specifier already has another use inside documentation, and then having a query alternative to use instead seems necessary. Exactly what this boils down to seems too complicated to explore here.

Copy link
Copy Markdown
Collaborator

@HansOlsson HansOlsson Oct 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A non-file system tool will still just need to manage one directory of "files": simply scrape the file system hierarchy for resources.d, and put these in a hierarchy that will make resolveURI easy.

I hadn't realized that the directories would be renamed as well, as default from Resources to resources.d.
To me that this looks like an unnecessary stylistic change, and I think this proposal is getting too many of them.

We clearly need to change from "modelica://Library/Resources/a.png" to "modelica:/Library/Resources/a.png" due to that part of the URI now being case-insensitive, and better fragments could be an improvement.

However, it's not clear that we need all the additional changes and everything is mixed together so it's hard to see which changes are needed and which ones are nice to have, etc. I would thus believe that we take a step back and first do the necessary changes and then optionally add the other ones.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A non-file system tool will still just need to manage one directory of "files": simply scrape the file system hierarchy for resources.d, and put these in a hierarchy that will make resolveURI easy.

I hadn't realized that the directories would be renamed as well, as default from Resources to resources.d.
To me that this looks like an unnecessary stylistic change, and I think this proposal is getting too many of them.

It is not unnecessary and just stylistic:

  • The name Resources is bad because it interferes with the storage of a class named Resources.
  • The directory name Resources is not reserved for storage of external resources.
  • External resources are not required to be stored in the directory Resources.
  • The name Resources is only standardized for external functions, not external resources in general.
  • The current rule The first part of the path shall not be the name of a class in the package given by the authority. is a cumbersome way of making the Modelica URI of an external resource unique.

We clearly need to change from "modelica://Library/Resources/a.png" to "modelica:/Library/Resources/a.png" due to that part of the URI now being case-insensitive, and better fragments could be an improvement.

However, it's not clear that we need all the additional changes and everything is mixed together so it's hard to see which changes are needed and which ones are nice to have, etc. I would thus believe that we take a step back and first do the necessary changes and then optionally add the other ones.

The reason to keep all of these related changes in one MCP is that we have a unique opportunity of making things better now that the new Modelica URIs will be distinguishable from the old ones.

The central part of the new Modelica URIs is that they try to really leverage the power of general URI structure. Keeping references to external resources in the form modelica:/Library/Resources/a.png would really get in the way of achieving this.

Note that a library would be able to keep its external resource in their current locations as long as they are intended to be accessed using the deprecate Modelica URIs. When the library author is ready to take the step to the new Modelica URIs, moving all external resources into a resources.d directory will be a trivial task.

If we were to remove the resources.d part entirely from this MCP, I would like to do it in such a way that it doesn't destroy the other aspects of the MCP. One way it could be done would be to use a different query than resource for accessing stuff stored in the old way:

  • modelica:/Modelica?filepath=Resources/a.png

While possible, I don't like the idea of introducing a filepath query, as it would make the transition harder to an improved storage model; to get rid of external resources residing anywhere in a library's directory structure, we'd need to deprecate also the filepath query in addition to the old Modelica URI form, with each deprecation being a nuisance to library authors.

@henrikt-ma henrikt-ma added the MCP0037 Generalized Modelica URIs (MCP-0037) label Sep 16, 2020
@HansOlsson
Copy link
Copy Markdown
Collaborator

Looking through this I additionally (in addition to the issues above that are still unresolved) see other choices that are questionable:

The choice between fragment, query, and nothing - the simple idea from descriptions of them is that 'fragment' should be part of the existing thing and query should be a query.

That does not seem to be applied consistently.

  • Using queries for figures is a good idea, as the figure doesn't exist as part of the model - but it can be generated. The query syntax would even allow parameter changes as part of the query.
  • On the other hand ?view=diagram and ?view=text seem like a bad change, they are fragments of the class. But Dymola supports more advanced "fragments" like #diagram$J1:flange_a - that should be a query (with a possible fragment, note that query precedes fragment in the URI-syntax) as it actually intends to do something (show sub-component and highlight something).
  • For resources there is similarly no need to involve a query.

To me a simple rule of thumb would be to look at the generated html-documentation. Fragments and resource-links correspond to fragments and links to corresponding resources (well, assuming the text is generated). Actual queries (figures and highlighting) would involve generating new contents as part of the html-generation, or link to a dynamic server.

@henrikt-ma
Copy link
Copy Markdown
Collaborator Author

Dear language group, I'd like to try moving this forward due to new needs to reference Modelica class related assets using URIs. Currently, we (Wolfram System Modeler) work around the problem using tool-specific URIs, but it is both impractical and feels wrong to do so when it is the basic structure of a new and improved Modelica URI we need.

I'd therefore like to raise the question if it's worth trying to make better progress here by dividing the MCP in two parts:

  1. The new form of Modelica URIs, with initial application only to the view query.
  2. The resource query along with the resolveURI operator.

We should then discuss how such a breakdown would be mapped to the MCP process.

Note that the figure query currently mentioned in the MCP documents is listed at all above, and would be saved as a future application of the new URIs.

@henrikt-ma henrikt-ma added this to the Phone 2023-5 milestone Nov 7, 2023
@HansOlsson
Copy link
Copy Markdown
Collaborator

As indicated above I don't find the query-syntax for these views ideal; and would reserve queries for cases where you have simulate/search for something.

If we think of statically generated html-documentation then I view fragments as static and being able to work across package-boundaries even if the pages are generated independently - whereas queries are "dynamic" and either require that the libraries are handled together, a dynamic server (actually doing queries), or will just not work.
Resources also fall in the static category IMO.

Having queries for some cases still make sense, but to me that would be additions to what we already have in Modelica (as indicated above having queries for selecting components in a model, going into sub-models etc).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

MCP0037 Generalized Modelica URIs (MCP-0037)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants